Codecov Report

Attention: Patch coverage is 60.86957% with 9 lines in your changes missing coverage. Please review.

Project coverage is 35%. Comparing base (4c75369) to head (2c2993c).
Report is 3 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (4c75369) and HEAD (2c2993c). Click for more details.

HEAD has 296 uploads less than BASE

Flag	BASE (4c75369)	HEAD (2c2993c)
Linux	59	9
torch2.0.1+cpu	18	3
python3.8	6	1
cpu	88	14
macOS	17	3
python3.12	18	3
torch2.5.0	6	1
python3.10	57	9
Windows	12	2
torch2.5.0+cpu	6	1
torch2.0.1	11	2
python3.11	7	1
torch2.4.1+cu121	15	2
torch2.2.2+cpu	6	1
torch2.1.2+cpu	6	1
torch2.3.1+cpu	6	1
torch2.5.0+cu124	14	2

@@           Coverage Diff            @@
##           master   #2806     +/-   ##
========================================
- Coverage      69%     35%    -34%     
========================================
  Files         338     338             
  Lines       18334   18352     +18     
========================================
- Hits        12586    6387   -6199     
- Misses       5748   11965   +6217     

@SkafteNicki thanks for implementing the fix! I just noticed another potential unexpected behaviour is when both targets and preds contain empty boxes:

targets = [{'boxes': torch.empty((0, 4), device='cuda:0'), 'labels': torch.tensor([], device='cuda:0', dtype=torch.int64)}]

preds = [{'boxes': torch.empty((0, 4), device='cuda:0'), 'labels': torch.tensor([], device='cuda:0', dtype=torch.int64), 'scores': torch.tensor([], device='cuda:0')}]

iou = IntersectionOverUnion()
iou.update(preds, targets)
result = iou.compute()  

This yields an IoU of 0, but it probably should return 1

@SkafteNicki thanks for implementing the fix! I just noticed another potential unexpected behaviour is when both targets and preds contain empty boxes:

targets = [{'boxes': torch.empty((0, 4), device='cuda:0'), 'labels': torch.tensor([], device='cuda:0', dtype=torch.int64)}]

preds = [{'boxes': torch.empty((0, 4), device='cuda:0'), 'labels': torch.tensor([], device='cuda:0', dtype=torch.int64), 'scores': torch.tensor([], device='cuda:0')}]

iou = IntersectionOverUnion()
iou.update(preds, targets)
result = iou.compute()  

This yields an IoU of 0, but it probably should return 1

@yurithefury I do partly not agree. When both target and preds contains empty boxes both the intersection and union of those boxes are 0, meaning that we end up trying to compute 0/0, which is undefined. It would maybe make the most sense setting it to Nan, but that would mess with average over multiple samples where only one is not defined. So there is really no right or wrong value in this case. Maybe it needs to be an argument to control?

@SkafteNicki Good point! Allowing an option to set this value, like empty_iou_value, would give flexibility - for example, setting it to NaN, 0, 1, or just ignoring these cases altogether. This way, users could pick what fits their use case best.